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earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 01 June 2005 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1 and 3-25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) 13 Claim(s) 1 and 3-25 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 
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10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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Attachment(s) 

1) O Notice of References Cited (PTO-892) 4) O Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Pa P er No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) L I Notice of Informal Patent Application (PTO-152) 

Paper No(s)/Mail Date . 6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 20050710 



Application/Control Number: 10/068,829 
Art Unit: 2176 



Page 2 



DETAILED ACTION 

1. This action is responsive to communications: RCE filed 6/1/2005, to the original application filed 
2/6/2002, drawings filed 3/25/2002. The present application is a continuation (filed under 37 CFR 1.53(b)) of 
parent case 08/900,421 (now abandoned). Said parent case filed 7/25/1997. 

2. Claims 1, 3-25 pending. Claims 21-25 have been added. Claims 1, 14, 16, 19, 20 are independent 
claims. 

Continued Examination Under 37 CFR 1.114 

3. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1. 17(e), 
was filed in this application after final rejection. Since this application is eligible for continued examination 
under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous 
Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/1/2005 has been 
entered. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 

rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the 
differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall 
not be negatived by the manner in which the invention was made. 

5. Claims 1, 6-13, 16-17, 19, 21, 23-24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mendelson et al. (hereinafter Mendelson), U.S. Patent No. 5,754,783 issued May 1998, in view of Durward 
et al. (hereinafter Durward), U.S. Patent No. 5,950,202 issued September 1999. 
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In regard to independent claim 1, Mendelson teaches interleaving (i.e. synchronizing) timed primary 
and untimed secondary data streams at a server, said resulting stream(s) eventually transported to clients for 
display (Mendelson Abstract, column 4 lines 18-22, column 8 lines 15-20). Mendelson defines timed primary 
content data as games, movies, audio music, etc., acquired and stored in a storage device (hard drive) on said 
server (Mendelson column 1 lines 13-22, column 6 lines 8-1 1, Figure 4 item 421). Mendelson defines untimed 
secondary content as video text, graphic images, private data such as files, databases, or stuffing bits, etc. 
(Mendelson column 1 lines 24-28, column 3 lines 50-54, column 6 lines 23-26, Figure 4 item 414). It is noted 
that any number of data streams can be utilized (Mendelson column 4 lines 54-58). 

Mendelson does not specifically teach motion "capture" data. However, Durward teaches a virtual 
reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since communication of 
changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion data stream is 
captured accordingly. It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply Durward to Mendelson, providing the benefit of motion data capture for a realistic virtual reality 
experience. 

Mendelson does not specifically disclose "motion" type data. However, Mendelson's secondary content 
(video text/images etc.) can at least be interpreted as suggestive of motion data since it is displayed as a 
video/movie type stream. In addition, Mendelson's secondary content can also comprise "padding", null/stuffing 
data, interleaved via a stuffing buffer, helping to preserve the quality (i.e. motion - wander, jitter, etc.) of the 
timed primary data (Mendelson column 2 lines 50-67, column 3 lines 9-20, 50-54, column 8 lines 14-17, Figure 
4 item 440). It would have been obvious to one of ordinary skill in the art at the time of the invention to interpret 
Mendelson's secondary data as a type of "motion" data, providing the benefit of controlling/minimizing possible 
timed motion related problems (i.e. wandering, jittering, etc.) of the primary content (compare the above two 
paragraphs with claim 1 "A method of synchronizing asynchronous time-based and motion data in a system in 
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which the time-based data and the motion data are transmitted by a server over a network to a client, the 
method comprising", and "retrieving a time-based data stream and a motion data stream at the server... ") 

Mendelson teaches receiving various time-based and motion data at a server, as explained above. In 
addition, since Mendelson discloses typical timed and untimed content within a frames environment (Mendelson 
column 1 lines 17-18, 24-25), combined with Mendelson's teaching of using the MPEG standard (typically 
applied to MPEG files, which are frame-based) (Mendelson column 4 lines 48-52), Mendelson's data streams 
can be interpreted as frame based (compare with claim 1 "... each stream comprising frames of data"). 

Mendelson teaches variable buffering (at a server) by measuring the bit rate stream, and buffers via the 
use of variable usage (interleaving) of primary, secondary and stuffing content buffers (Mendelson column 6 
lines 62-67, column 8 lines 6-20, Figure 4 items 440, 450, 460). It is noted that each type of content has its own 
buffer, and are regulated accordingly. It is noted that Mendelson's resulting data stream can be interpreted as 
comprising at least two interleaved streams (now custom synchronized) of various data types. In addition, 
multiple elementary streams within a transport stream can be utilized (Mendelson column 4 lines 54-63, column 
6 lines 4-8) (compare with claim 1 "variable buffering one of the time-based data stream and the motion data 
stream at the server to produce two streams having synchronized frames"). 

Mendelson teaches transport of a final (custom synchronized) stream(s) to customers (clients) for 
eventual playback (Mendelson column 6 lines 4-8, column 8 lines 10-12; compare with claim 1 "using the 
synchronized frames at the client for playback of synchronized motion and time-based data to a user."). 

In regard to dependent claim 6, Mendelson teaches secondary data as stuffing data, or as private data 
such as files or databases (see rejection of claim 1, also Mendelson column 6 lines 24-26). Mendelson does not 
specifically teach said motion data mapped to control movement of a virtual figure displayed in a scene at the 
client. However, Durward teaches a method whereby updated positional data from a person's head position 
sensor is mapped and used to determine the position of a virtual being defined for that user, communicating 
graphical data to the user via database data (Durward, column 6 lines 29-32, 46-52, column 7 lines 12-20). It 
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would have been obvious to one of ordinary skill in the art at the time of the invention to apply the virtual 
position database data of Durward to Mendelson's secondary database data, providing Mendelson increased 
spatial imagery to the multimedia stream 

In regard to dependent claim 7, Mendelson does not specifically teach using a body suit for 
generating motion data. However, Durward teaches a method of sensing position data with the use of a body suit 
(Durward, column 3 line 25-33). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to apply Durward to Mendelson, providing Mendelson the benefit of three dimensional physical 
contour mapping, providing increased spatial image accuracy to Mendelson's video stream method. 

In regard to dependent claim 8, Mendelson teaches primary content (video, audio data, etc.) as well as 
secondary content (video text, images, files, database data, along with "stuffing" data, etc.) as explained in the 
rejection of claim 1. As the resulting interleaved stream(s) is played back on a client, any combination of 
foregroundfaackground combination can be typically specified, (i.e. the primary data (i.e. a movie scene) can be 
run as background to the secondary/motion data controlling it, or run as background to any related database files 
in the foreground, etc). 

In regard to dependent claims 9, 10, Mendelson teaches buffering/processing occurring at the same 
time the program is being transported to customer equipment (Mendelson column 8 lines 9-12; compare with 
claim 9). 

Mendelson teaches audio (i.e. voice) data (Mendelson column 4 lines 57-59; compare with claim 10). 

In regard to dependent claim 11, Mendelson teaches an embodiment using streams associated with 
data channels (Mendelson column 6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 
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Mendelson does not specifically teach that a client may join in progress a multicasting of a stream 
However, Durward teaches virtual reality presentations where a number of contributing user's data is collected 
and analyzed accordingly, so a to facilitate multicasting (and interaction between figures) of the virtual world to 
various users (Durward Figures 4-6, 7 column 8 lines 5-15). Durward also teaches users logging on to a system 
via modems, said users data is shared with other users and continuously updated so as to create a virtual reality 
world (Durwald, column 8 lines 22-24, 41-64, Figure 1). It would have been obvious to one of ordinary skill in 
the art at the time of the invention to apply Durward to Mendelson, providing Mendelson the benefit of 
multicasting associated with synchronization of streams. 

In regard to dependent claims 12, 13, Mendelson teaches buffering of streams comprising buffering of 
primary and secondary content (said primary content can be audio data, Mendelson column 4 lines 57-59), along 
with necessary interleaved motion type data (secondary/stuffing data), producing a sequenced synchronized 
stream regarding constant verses variable bit stream rate (see also rejection of claim 1, also Mendelson column 6 
lines 62-67). 

In regard to independent claim 16, claim 16 reflects the apparatus used to perform the methods 
comprising computer readable instructions as claimed in claim 1, and in further view of the following, is 
rejected along the same rationale. 

Mendelson does not specifically teach motion "three dimensional" data stream. However, Durward 
teaches a virtual reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since 
communication of changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion 
data stream is captured accordingly and is associated with 3D data. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing the benefit of 
motion data 3D capture for a realistic virtual reality experience. 
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In regard to dependent claim 17, Mendelson does not specifically teach an apparatus whereby 
synchronized motion/time-based data is milticasted to clients in a network. However, Durward teaches an 
apparatus whereby users log on to a system via modems, said users data is shared with other users and 
continuously updated so as to create a virtual reality world (Durward, column 8 lines 22-24, 41-64, Figure 1). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply the data sharing 
method of Durward to Mendelson, providing Mendelson the benefit of continuous updating and increased 
uniform user awareness in a virtual environment. 

In regard to independent claim 19, claim 19 incorporates substantially similar subject matter as 
claimed in claim 1, and in further view of the following, is rejected along the same rationale. 

Mendelson does not specifically teach motion "capture' data. However, Durward teaches a virtual 
reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since communication of 
changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion data stream is 
captured accordingly. It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply Durward to Mendelson, providing the benefit of motion data capture for a realistic virtual reality 
experience. 

Mendelson teaches secondary data as stuffing data, or as private data such as files or databases (see 
rejection of claim 1, also Mendelson column 6 lines 24-26). Mendelson does not specifically teach said motion 
data mapped to control movement of a virtual figure displayed in a scene at the client, along with 
synchronization. However, Durward teaches a method whereby updated positional data from a person's head 
position sensor is mapped and used to determine the position of a virtual being defined for that user, 
communicating graphical data to the user via database data (Durward, column 6 lines 29-32, 46-52, column 7 
lines 12-20). Since the playback of Mendelson's resulting stream synchronizes the secondary data with the 
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primary content, the primary data (i.e. video) can be synchronized with added secondary database data (related 
to a virtual figure). It would have been obvious to one of ordinary skill in the art at the time of the invention to 
apply the virtual position database data of Durward to Mendelson's secondary database data, providing 
Mendelson increased spatial imagery and desired synchronization to the resulting multimedia stream 

In regard to dependent claim 21, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 

In regard to dependent claim 23, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 

In regard to dependent claim 24, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
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obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 

6. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable Mendelson and Durward as 
applied to claim 1 above, and further in view of Baumgartner et al. (hereinafter Baumgartner), U.S. 
Patent No. 5,642,171 issued June 1997. 

In regard to dependent claim 3, Mendelson teaches buffering based on a calculated difference (delay) 
between a variable bit stream, and a constant bit stream (Mendelson column 6 lines 62-67). Mendelson does not 
specifically teach a difference between the delay for motion and time-based data stream to determine buffering 
for a faster stream However, Baumgartner teaches a method whereby a current video frame number is 
subtracted from a current audio frame number to determine if the audio is too far ahead of the video 
(Baumgartner column 13 lines 60-67, column 14 lines 1-2). It would have been obvious to one of ordinary skill 
in the art at the time of the invention to apply the difference calculation method of Baumgartner to Mendelson' s 
stream analysis, providing Mendelson the benefit of more accurate time synchronization to the server buffer 
method disclosed by Mendelson. 

7. Claims 4, 14-15, 18, 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Mendelson 
and Durward, and further in view of Shibata, Y., Media synchronization protocols for packet audio-video 
system on multimedia information networks (hereinafter Shibata), IEEE; January 3-6, 1995 pp.594-601. 

In regard to dependent claim 4, Mendelson does not specifically teach transfer of data values for a 
frame occuring only for a frame that has changed since a last frame was transmitted. However, Shibata discloses 
a method whereby audio data is sent from the video server to the client station only during a "talk spurt", with 
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constant frame rates occurring during periods of audio silence (Shibata p. 597, section 4.3 "Silence detected 
synchronization"). It would have been obvious to one of ordinary skill in the art at the time of the invention to 
apply Shibata to Mendelson's stream interleaving, providing Mendelson the benefit of file size and transmission 
reduction (streamlining) of the audio/video data streams. 

In regard to independent claim 14, claim 14 incorporates substantially similar subject matter as 
claimed in claim 1, and in further view of the following, is rejected along the same rationale. 

Mendelson does not specifically teach motion "three dimensional" data stream. However, Durward 
teaches a virtual reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since 
communication of changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion 
data stream is captured accordingly and is associated with 3D data. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing the benefit of 
motion data 3D capture for a realistic virtual reality experience. 

Mendelson teaches an embodiment using streams associated with "data channels" (Mendelson column 
6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), said header typically used to describe 
data within transport stream data (i.e. each 188 byte packet) (Mendelson column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach transfer of data values (channels) for a occuring only for changed 
data values. However, Shibata discloses a method whereby audio data is sent from the video server to the client 
station only during a "talk spurt", with constant frame rates occurring during periods of audio silence (i.e. a talk 
spurt changes the data values) (Shibata p. 597, section 4.3 "Silence detected synchronization"). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to apply Shibata to Mendelson's 
stream interleaving, providing Mendelson the benefit of file size and transmission reduction (streamlining) of 
the audio/video data streams. It is to be noted that in order to adjust frame rates as above, the server must know 
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that a change in the data has occurred, which is typically achieved by comparing new data (the talk spurt) with 
the last recognized data value (previously recognized as empty - reflecting a period of silence). 

In regard to dependent claim 15, Mendelson teaches an embodiment using streams associated with 
data channels (Mendelson column 6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

In regard to dependent claim 18, Mendelson teaches an embodiment using streams associated with 
"data channels" (Mendelson column 6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach transfer of data values (channels) for a occuring only for changed 
data values. However, Shibata discloses a method whereby audio data is sent from the video server to the client 
station only during a "talk spurt", with constant frame rates occurring during periods of audio silence (i.e. a talk 
spurt changes the data values) (Shibata p. 5 97, section 4.3 "Silence detected synchronization"). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to apply Shibata to Mendelson's 
stream interleaving, providing Mendelson the benefit of file size and transmission reduction (streamlining) of 
the audio/video data streams. It is to be noted.that in order to adjust frame rates as above, the server must know 
that a change in the data has occurred, which is typically achieved by comparing new data (the talk spurt) with 
the last recognized data value (previously recognized as empty - reflecting a period of silence). 

In regard to dependent claim 22, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
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obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 

8. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Mendelson and Durward, 
as applied to claims 1, above, and further in view of Katseff et al. (hereinafter Katseff), U.S. Patent No. 
5,822,537 issued October 1998. 

In regard to dependent claim 5, Mendelson teaches a networking environment (Mendelson column 4 
lines 28-33). Mendelson does not specifically teach using the Internet for distribution. However, Katseff teaches 
an internet as an example of suggested types of networks that can be used (Katseff, column 3 lines 58-63). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply Katseff to 
Mendelson, providing Mendelson the benefit of a widespread and familiar network. 

10. Claims 20, 25 are rejected under 35 U.S.C. 103(a) as being unpatentable over Mendelson, 
Durward, and Baumgartner. 

In regard to independent claim 20, claim 20 incorporates substantially similar subject matter as 
claimed in claims 1, 3, 10, 1 1, and is rejected along the same rationale. 

In regard to dependent claim 25, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 
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Response to Arguments 



1 1 . Applicant's arguments filed 6/1/2005 have been fully and carefully considered but they are not 
persuasive. 

Applicant's arguments are substantially directed towards amended subject matter. The examiner relies 
upon Durward to teach the amended limitations regarding "3D" and motion "capture", since Durward is a three 
dimensional virtual reality system utilizing helmet sensor data. 

It is respectfully submitted that the various combination of references teach Applicant's claimed 
invention as currently claimed. 



12. Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to William L. Bashore whose telephone number is (571) 272-4088. The examiner can normally be 
reached on 1 1 :30am - 8:00pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Heather 
Herndon can be reached on (571) 272-4136. The fax phone number for the organization where this application 
or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system Status information for published applications may be obtained from 
either Private PAIR or Public PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 



have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 
9197 (toll-free). 



Conclusion 




